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Abstract 

The Renyi entropy is a generalization of the usual concept of entropy which depends on a 
parameter q. In fact, Renyi entropy is closely related to free energy. Suppose we start with 
a system in thermal equilibrium and then suddenly divide the temperature by q. Then the 
maximum amount of work the system can do as it moves to equilibrium at the new temperature, 
divided by the change in temperature, equals the system's Renyi entropy in its original state. 
This result applies to both classical and quantum systems. Mathematically, we can express 
this result as follows: the Renyi entropy of a system in thermal equilibrium is minus the 'q~^- 
derivative' of its free energy with respect to temperature. This shows that Renyi entropy is a 
q-deformation of the usual concept of entropy. 

1 Introduction 

In 1960, Renyi [13] defined a generalization of Shannon entropy which depends on a parameter. If 
p is a probability distribution on a finite set, its Renyi entropy of order q is defined to be 

i 

where < q < oo. Of course we need g 7^ 1 to avoid dividing by zero, but L'Hopital's rule shows 
that the Renyi entropy approaches the Shannon entropy as q approaches 1: 



lim 5„ = - y^p. In Pi. 

(7— >1 ^ ^ 



So, it is customary to define S'l to be the Shannon entropy. 

While Shannon entropy has a deep relation to thermodynamics, the Renyi entropy has not 
been completely integrated into this subject — at least, not in a wcU-rccognizcd way. While many 
researchers have tried to modify statistical mechanics by changing the usual formula for entropy, 
so far the most convincing uses of Renyi entropy in physics seem to involve the limiting cases 
So = limg_^o and 6*00 = ^^niq^+oo Sq. These are known as the 'max-cntropy' and 'min-entropy', 
respectively, since 5*^ is a decreasing function of q. They show up in studies on the work value of 
information [6] and the thermodynamic meaning of negative entropy [2]. For other interpretations 
of Renyi entropy see Harremoes [9], Konig et al. [10], and Uffink [14]. 

In fact, it is not necessary to modify statistical mechanics to find a natural role for Renyi entropy 
in physics. Renyi entropy is closely related to the familiar concept of free energy, with the parameter 
q appearing as a ratio of temperatures. 
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The trick is to think of our probability distribution as a Gibbs state: that is, the state of 
thermal equilibrium for some Hamiltonian at some ehosen temperature, say Tq. Suppose that all 
the probabilities pi are nonzero. Then working in units where Boltzmann's constant equals 1, we 
can write 

ft = e-^'^^" 

for some nonnegative real numbers Ei. If we think of these numbers as the energies of microstates of 
some physical system, the Gibbs state of this system at temperature T is the probability distribution 



where Z is the partition function: 



Z 



Since Z = 1 when T = Tq, the Gibbs state reduces to our original probability distribution p at this 
temperature. 

Starting from these assumptions, the free energy 

F = -TlnZ 

is related to the Renyi entropy as follows: 

F = -{T~To)St,/t (2) 

The proof is an easy calculation: 

Sn/T = ^InV pf"/^ ^ e-^'/^ = —. 

^"'^ l-To/T r-To ^ T-To 

This works for T Tq, but we can use L'Hopital's rule to show that in the limit T — > Tq both sides 
converge to the Shannon entropy 5*1 of the original probability distribution p. 

After the author noticed this result in the special case Tq = 1 [3] , Stacey commented that this case 
was already mentioned in Beck and Schlogl's 1995 text on the thermodynamics of chaotic systems [4]. 
However, most people using Renyi entropy seem unaware of its connection to free energy, perhaps 
because they work on statistical inference rather than physics [8]. The connection deserves to be 
better known. 

Shortly after the first draft of this paper was released, Polettini pointed out the generalization 
to other values of To [11]. He also gave a nice physical intepretation of Renyi entropy [12]. Downes 
then made a further generalization [7]. The above argument concerns a system with Gibbs state 
Pi — ex'p{—Ei/To) at a chosen temperature Tq. Such a system automatically has zero free energy at 
this chosen temperature. Downes generalized the relation between Renyi entropy and free energy to 
systems whose free energy is not constrained this way. Polettini's physical interpretation of Renyi 
entropy can be extended to these more general systems, and we describe this interpretation in what 
follows. We also note that Renyi entropy is a 'g-deformation' of the ordinary notion of entropy. This 
complements the work of Abe on another generalization of entropy, the Tsallis entropy [1]. 

In what follows, we work in a quantum rather than classical context, using a density matrix 
instead of a probability distribution. However, we can diagonalize any density matrix, and then its 
diagonal entries define a probability distribution. Thus, all our results apply to classical as well as 
quantum systems. The quantum generalization of Shannon entropy is, of course, well-known: it is 
often called 'von Neumann entropy'. The quantum generalization of Renyi entropy is also already 
known [15]. 
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2 Renyi Entropy as a g-Derivative of Free Energy 

Let iJ be a self-adjoint complex matrix. Thinking of H as the Hamiltonian of a quantum system, 
and working in units where Boltzmann's constant is 1, we may define the Gibbs state of this system 
at temperature T to be the density matrix 

where the partition function 

Z{T) = tr(e--^/'^) (4) 
ensures that tr(pT) = 1. The free energy at temperature T is defined by 

F(T) = -TlnZ(T). (5) 

On the other hand, for any density matrix p, the quantum generalization of Renyi entropy is 
defined by 

SM ^ (6) 

l~q 

since this formula reduces to the usual definition of Renyi entropy. Equation (1), when the proba- 
bilities Pi are the eigenvalues of p. This formula makes sense when < g < oo and q I, but we 
can define the quantum Renyi entropy as a limit in the special cases q = 0,1, +oo. For q = 1 this 
gives the usual von Neumann entropy: 

Slip) ■= lim Sq{p) = -tr(plnp). (7) 

Returning to our system with Gibbs state pT at temperature T, let us write Sq{T) for Sq{pT)- 
Gomputing this Renyi entropy at some temperature Tq, we find: 

1 1 fe-<i"l'^° 

SniTn) = Intrfp!^, ) = Intr , — 

^' l-q ^^^'>' 1-q V Z{TnY 

If we define a new temperature T with 

q = To/T, (8) 

we obtain: 

InZ(T) - q\uZ{To) 



SqiTo) 



l-q 

T\nZ{T) - rolnZ(ro) 
T-Tn 



or in short: 



To IT 



(To) 



F{T) - F{n) 



(9) 



T-To 

This equation, noted by Downes [7] upon reading the first draft of this paper, is the clearest way 
to express the relation between Renyi entropy and free energy. In the special case where the free 
energy vanishes at temperature Tq, it reduces to Equation (2). In the limit T — > Tp, it reduces to 



Si{To) 



dF{T) 
dT 



(10) 



T=To 
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Of course, it is already well-known that the von Neumann entropy is the derivative of —F with 
respect to temperature. What we see now is that the Renyi entropy is the difference quotient 
approximating this derivative. Instead of the slope of the tangent line, it is the slope of the secant 
line. 

In fact, we can say a bit more: the Renyi entropy is the the '(/"^-derivative' of the negative free 
energy. For q ^ 1, the g-derivative of a function / is defined by 



This reduces to the ordinary derivative in the limit g — 1. The ^-derivative is defined the same 
way but with replacing q. Equation (9) can be rewritten more tersely using this concept as 
follows: 



Here we have made a change of variables, writing T for the variable called To in Equation (9). 

The concept of g-derivative shows up in mathematics whenever we 'g-deform' familiar structures, 
obtaining new ones such as quantum groups. For an introduction, see the text by Cheung and Kac 
[5]. In some cases g-deformation should be thought of as quantization, with q playing the role of 
exp(?i). That is definitely not the case here: the parameter q in our formulas is unrelated to Planck's 
constant h. Indeed, Equation (11) holds in classical as well as quantum mechanics. 

What; then, is the thermodynamic meaning of Renyi entropy? This was nicely explained by 
Polettini [12]. Start with a physical system in thermal equilibrium at some temperature. Then 
'quench' it, suddenly dividing the temperature by q. The maximum amount of work the system can 
do as it moves to thermal equilibrium at the new temperature, divided by the change in temperature, 
equals the system's Renyi entropy of order q in its original state. Note that this formulation even 
accounts for the minus sign in Equation (9), because it speaks of the work the system does, rather 
than the work done to it. 
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